Complete the Anvil-to-Tevm test migration - #2

Review: correctness of the migration itself

Read at f8618fe (291 files, +16173/-3015). I focused on the question "does a passing test still prove what it proved before?" rather than on style. Five things below are, I think, genuine defects rather than known-issue triage. Everything I claim to have run is pasted verbatim.

1. Every file that touches the harness now hard-requires an archive RPC — including tests that need no chain state

test/setup.ts gates fork init on tevmMainnet.isUsed(), and isUsed() is driven by a Proxy in test/src/tevm.ts whose trap flips used = true on any property read except isUsed itself:

get(target, property, receiver) {
  if (property !== 'isUsed') used = true
  return Reflect.get(target, property, receiver)
}

A module-level const client = tevmMainnet.getClient() — the single most common line in this diff — therefore forces a real network fork in beforeAll, even when no test in the file ever sends a request to the node. test/src/utils.ts makes this worse: it reads tevmMainnet.forkUrl at module scope, so every file importing ~test/utils.js is marked used too.

Concretely, on this branch with a default (non-archive) fallback endpoint:

$ pnpm exec vitest run -c test/vitest.config.ts --project core \
    src/actions/wallet/requestAddresses.test.ts src/utils/signature/verifyTypedData.test.ts

URL: https://cloudflare-eth.com
Request body: {"method":"eth_getBlockByNumber","params":["0x153b745",false]}
Details: Internal error
...
 Test Files  2 failed (2)
      Tests  3 skipped (3)
     Errors  4 errors
   Duration  13.06s

Neither of those files needs fork state. requestAddresses is answered by the harness's own stub; verifyTypedData's default case is local ECDSA. Proof — same two files, same commit, fork disabled:

$ VITE_TEVM_NO_FORK=true pnpm exec vitest run -c test/vitest.config.ts --project core \
    src/actions/wallet/requestAddresses.test.ts src/utils/signature/verifyTypedData.test.ts

 Test Files  2 passed (2)
      Tests  2 passed | 1 skipped (3)
   Duration  17.10s

This is not "archive-provider rate limits". It is the laziness mechanism not being lazy: isUsed tracks reference, not use. Under Anvil the equivalent files were cheap because the pooled node was spawned once per worker and reset with setIntervalMining, not re-forked per file. A meaningful slice of the 232 reported runtime errors is likely this, and it is a migration bug, not a Tevm gap.

Suggested fix: mark used on the transport request (inside custom({ async request(...) })) rather than on property access, or expose an explicit harness.use() that getClient() calls while plain metadata reads (forkUrl, chain, forkBlockNumber) do not.

2. setup.ts only resets tevmMainnet. The OP-Stack, Sepolia and ZKsync harnesses are never reset — ever

if (process.env.SKIP_GLOBAL_SETUP || !usesTevm || !tevmMainnet.isUsed()) return
await tevmMainnet.reset()

tevmOptimism, tevmSepolia and tevmZksync are module-level singletons in test/src/tevm.ts. Nothing snapshots or reverts them between tests or between files in the same worker. On main, setup.global.ts started one Anvil instance per exported fixture and the pooled proxy gave each worker a fresh node, and test/setup.ts reset mining state per test.

So today: a ZKsync or OP-Stack test that mutates balances, nonces or storage leaks that state into every later test in the same worker. Those suites become order-dependent, and with retry: 3 in test/vitest.config.ts a leak-induced failure can be masked on retry or, worse, a retry can pass against dirty state. This changes what those tests prove.

3. Tests call anvil_reset on the shared harness, which invalidates the baseline snapshot the whole file depends on

test/src/tevm.ts holds one baselineSnapshot and reset() does evm_revert(baselineSnapshot), throwing Failed to revert Tevm snapshot ... if the node says no. But 19 migrated test files call viem's reset() action against the shared client, e.g. src/actions/public/estimateGas.test.ts:

await reset(client, {
  blockNumber: tevmMainnet.forkBlockNumber,
  jsonRpcUrl: tevmMainnet.forkUrl,
})

and the whole OP-Stack suite (getL2Output, getGame, getWithdrawalStatus, proveWithdrawal, waitToFinalize, …).

The PR clearly knows this hazard — src/clients/createTestClient.test.ts was specially rewritten to spin up an isolated harness for exactly reset and setRpcUrl:

// `anvil_reset` invalidates the shared harness's baseline snapshot, so
// this test runs against an isolated Tevm node.
const harness = createTevmTestHarness()

The same reasoning applies to all 19 files above and wasn't applied there. This is a plausible mechanical source of the reported InvalidBlockError "fork divergences": after an in-test anvil_reset, the next beforeEach reverts to a snapshot taken against a different fork anchor. I'd want that hypothesis excluded before any of those failures is written off as a Tevm gap.

Cheap structural fix: have harness.reset() re-take the baseline if evm_revert returns false, and/or add an afterEach that re-anchors. Better: make createTevmTestHarness() per-file the default and drop the shared singleton.

4. Anvil is intentionally opt-in now, but the opt-in lane it was replaced with does not actually work for IPC

test/src/anvil-transports.ts advertises rpcUrl.ipc = '/tmp/viem-anvil-transports.ipc', but start() never passes it to Anvil:

const instance = Instance.anvil({
  chainId: mainnet.id,
  forkBlockNumber: this.forkBlockNumber,
  forkUrl: this.forkUrl,
})

No ipcPath/--ipc, so src/clients/transports/ipc.test.ts and src/utils/rpc/ipc.test.ts have no socket to connect to. Also dropped versus main's anvilMainnet: noMining: true. Those suites were written against a non-auto-mining node; on this fixture Anvil auto-mines. So the claim that these four suites "remain unchanged in substance" doesn't hold — the fixture underneath them changed, and since the lane is excluded from CI nothing catches it. Worth running TEST_ANVIL_TRANSPORTS=true once and pasting the result, otherwise this lane is dead code from day one.

Review, part 2: coverage silently removed, and triage of the "known failures"

5. The WebSocket half of the local test matrix was deleted from CI, and this isn't in the PR description

.github/workflows/verify.yml:

       matrix:
         multicall: ['true', 'false']
-        transport-mode: ['http', 'webSocket']
         shard: [1, 2, 3]
...
-          VITE_NETWORK_TRANSPORT_MODE: ${{ matrix.transport-mode }}

That leg did not run "four WebSocket suites" — it ran the entire local test suite a second time with every client's transport switched to WebSocket. Losing it is a far bigger coverage loss than the four opt-in files, and the PR description frames the WebSocket gap as only those four files. It should be called out explicitly as part of the blocker, because it's the thing that has to come back when @tevm/server WS lands.

Related, three more tests were moved out of default CI without being listed: webSocket - getRpcClient()/subscribe()/request() - keepAlive & reconnect disabled moved from src/clients/createPublicClient.test.ts into src/clients/transports/webSocket.test.ts, which is in the excluded lane. Net: 4 files + 3 tests + the whole WS matrix leg.

6. CI now passes one fork secret where it used to pass five

-          VITE_ANVIL_FORK_URL: ...
-          VITE_ANVIL_FORK_URL_OPTIMISM: ...
-          VITE_ANVIL_FORK_URL_OPTIMISM_SEPOLIA: ...
-          VITE_ANVIL_FORK_URL_SEPOLIA: ...
-          VITE_ANVIL_FORK_URL_ZKSYNC: ...
+          VITE_TEVM_FORK_URL: ${{ secrets.VITE_TEVM_FORK_URL }}

test/README.md documents VITE_TEVM_FORK_URL_OPTIMISM, _SEPOLIA and _ZKSYNC, and test/src/tevm.ts reads them — but CI never sets them, so in CI the Optimism fixture always falls back to https://mainnet.optimism.io. That public endpoint has a ~10,000-block state window, and the fixture is pinned at block 113624777. The PR lists "the available Optimism endpoint rejecting historical eth_getProof outside its 10,000-block window" as an external constraint. It's a wiring omission: wire the three secrets and that class of failure should go away. Same for the Sepolia/ZKsync suites.

(For the record, the cloudflare-eth.com / rpc.sepolia.org fallbacks are not a regression — main:test/src/anvil.ts had identical fallbacks. Only the CI secret wiring regressed.)

7. Foundry: it's now fully gone, which is more than the description says — and the generated artifacts are unguarded

contracts:build is deleted from package.json, removed from postinstall, removed from two CI jobs, forge removed from the knip ignoreBinaries, generated.ts un-gitignored, and 8,348 lines of contracts/generated.ts checked in. That's a good outcome (no Foundry needed to install or to run CI), but two consequences aren't addressed:

• The .sol sources and the lib/ submodule remain, and there is now no check that contracts/generated.ts matches them. Edit a contract, forget to regenerate, and nothing anywhere fails. Previously postinstall regenerated on every install. A contracts:check CI job (build with forge, git diff --exit-code contracts/generated.ts) would restore the guarantee without putting Foundry back in the default install path.
• There's no documented command left for regenerating them at all, since the script itself was removed. scripts/generateTypedArtifacts.ts is now unreachable from any package script.

8. Triage of the described remaining failures

Splitting them, because the PR's list mixes categories:

9. Smaller notes

• createCommon({ ..., customCrypto: { kzg: createMockKzg() } }) in test/src/tevm.ts means the node performs no real KZG verification. The active 4844 tests are test.skipped today so nothing is currently mis-asserting, but when args: blobs / the 4844 cases are unskipped they'd pass against a mock verifier. Worth a comment at the mock so nobody unskips them and believes the result.
• src/clients/createTestClient.test.ts creates two extra createTevmTestHarness() instances (reset, setRpcUrl) with no teardown. The harness exposes no dispose/close at all, so each leaks a MemoryClient and its fork transport for the life of the worker. Adding harness.close() (and calling it in afterAll) is cheap now and gets much harder later.
• src/actions/wallet/requestAddresses.test.ts still snapshots one address, but the harness now answers eth_requestAccounts from a hardcoded [accounts[0].address] rather than from the node. The snapshot is unchanged from main so nothing regressed, but the test no longer exercises anything.
• const client = anvilMainnet.getClient() as never in webSocket.test.ts casts away the client type for the whole file. Prefer widening the fixture's return type.

Nothing here needs to block the direction — the migration is clearly the right move and the honest red report is appreciated. But #1, #2, #3 and wevm#5 change what passing tests prove, and I'd want those separated from the Tevm-gap list before the red run is treated as a baseline. Follow-up scope for finishing the job now that tevm WS (wevm#2084) and IPC (wevm#2080) are merged is in a separate comment.

Follow-up scope: retiring the Anvil lane once tevm WS/IPC publish

evmts/tevm wevm#2084 (createWebSocketServer, JSON-RPC over WS on the same port as HTTP, eth_subscribe/eth_unsubscribe with real notification push) and wevm#2080 (createIpcServer, Unix domain socket, framing across fragmented/concatenated/newline-delimited chunks, newHeads/newPendingTransactions/logs subscriptions, cleanup on eth_unsubscribe and socket close) are both merged into evmts/tevm main. The stated blocker for this PR's Anvil lane is therefore gone in source — but not on npm: publishing is blocked on npm trusted-publisher configuration, and this fork pins tevm@1.0.0-rc.151 from the registry. So this is sequenced, not immediately actionable.

Phase 0 — do now, independent of publishing

None of this waits on tevm:

1. Fix isUsed so it tracks transport requests, not property reads (review comment test: add an in-process Tevm test lane, replacing Anvil where rc.151 supports it #1). Biggest single win; likely removes most of the 429s and a large share of the 232 runtime errors.
2. Reset (or isolate) tevmOptimism / tevmSepolia / tevmZksync in test/setup.ts (Complete the Anvil-to-Tevm test migration #2).
3. Give the 19 files that call reset() their own harness, or make harness.reset() re-anchor when evm_revert returns false (📝 docs(test): follow-up scope for finishing the Anvil-to-Tevm migration #3).
4. Wire VITE_TEVM_FORK_URL_OPTIMISM / _SEPOLIA / _ZKSYNC secrets into verify.yml (docs: sidebar scaffold wevm/viem#6).
5. Give Alto its own non-snapshot-reverted harness in test/src/bundler.ts (feat: polish websocket rpc implementation wevm/viem#8).
6. Re-record snapshots against the new 22263621n anchor as a separate mechanical commit, then measure what behavioural drift actually remains.
7. Add a contracts:check CI job so contracts/generated.ts can't silently go stale (feat: fetchTransactionReceipt wevm/viem#7).
8. Fix test/src/anvil-transports.ts: pass an ipcPath to Instance.anvil and restore noMining: true, then actually run TEST_ANVIL_TRANSPORTS=true in a scheduled (non-blocking) CI job so the lane doesn't rot while it exists (refactor: convert blockNumber to bigint wevm/viem#4).

Phase 1 — when a tevm release containing wevm#2084/wevm#2080 is on npm

Bump tevm/@tevm/* off 1.0.0-rc.151 (also drop the pnpm.packageExtensions viem-injection hack in package.json if the new release ships a compatible viem peer).

Then extend test/src/tevm.ts with a withServers() variant that starts, on one harness:

• createServer (HTTP, already used by test/src/bundler.ts),
• createWebSocketServer on the same port,
• createIpcServer on a per-worker socket path (/tmp/viem-tevm-${VITEST_POOL_ID}.ipc),

exposing rpcUrl.{http,ws,ipc} with the same shape anvil-transports.ts fakes today, plus a real close(). Ports must be VITEST_POOL_ID-offset like test/src/bundler.ts already does; the IPC path must be unlinked in teardown.

The four suites, and exactly what each needs

Once those four are on the tevm servers: delete test/src/anvil-transports.ts, test/setup.anvil-transports.global.ts, the anvilTransportTests array and the conditional anvil-transports project in test/vitest.config.ts, and the TEST_ANVIL_TRANSPORTS branch in test/setup.ts.

Then restore the WebSocket CI matrix leg

The transport-mode: ['http', 'webSocket'] axis and VITE_NETWORK_TRANSPORT_MODE deleted from verify.yml must come back pointed at the tevm WS server. This is the largest piece of coverage the migration dropped and it is only recoverable at this phase. Note test/src/tevm.ts currently has no VITE_NETWORK_TRANSPORT_MODE handling at all — the harness always returns the in-process custom() transport — so getClient() needs a mode switch that returns webSocket(rpcUrl.ws) when the env var says so.

What's left before Anvil is fully gone

After the above, the residue is:

• prool — still needed for Alto (and the Tempo fixture), not for Anvil. It stays as a devDependency but no longer starts an Ethereum node. Removing the Alto dependency is a separate, larger project (an in-process bundler), not part of this.
• ~70 files with anvil in them that are public API, not a node: src/chains/definitions/anvil.ts, mode: 'anvil' in createTestClient, the anvil_* method spellings in src/actions/test/*, error mappings in src/errors/node.ts, docs. These are viem's supported surface and Tevm's test-RPC dialect. They should stay, and the PR description is right to separate them — but the migration would read better if the harness's mode: 'anvil' as const had a one-line comment at each site (it has one, in getClient(); the createTestClient call sites don't).
• Foundry: separate from Anvil, and this PR already removes it from postinstall and CI by checking in contracts/generated.ts. The remaining tie is the lib/ submodule + .sol sources with no regeneration path (feat: fetchTransactionReceipt wevm/viem#7). Restoring a contracts:build script gated behind a contracts:check CI job is the right end state: Foundry needed only by contract authors, never by installers or CI test runs.

Definition of done

• pnpm test green with no TEST_ANVIL_TRANSPORTS project defined and no anvil binary on PATH.
• CI matrix has both http and webSocket legs again.
• grep -rl 'prool' test/ returns only Alto/Tempo fixtures.
• test.skips reduced to the three genuine Tevm gaps (eth_sendUnsignedTransaction, eth_signTypedData_v4, EIP-4844 fork anchor), each linked to an upstream evmts/tevm issue.